COSTA MT Evaluation Tool: An Open Toolkit for Human Machine Translation Evaluation

COSTA MT Evaluation Tool: An Open Toolkit for Human Machine Translation Evaluation

Ahotly debated topic inmachine translation is human evaluation. On the one hand, it is extremely costly and time consuming; on the other, it is an important and unfortunately inevitable part of any system. This paper describes COSTAMT Evaluation Tool, an open stand-alone tool for human machine translation evaluation. It is a Java program that can be used to manually evaluate the quality of the ...

Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation

This article describes the A Toolkit for Automatic Machine Translation Evaluation and Meta-evaluation, an open framework offering system and metric developers a text interface to a rich repository of metrics and meta-metrics.

An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research

In this paper we present a tool for the evaluation of translation quality. First, the typical requirements of such a tool in the framework of machine translation (MT) research are discussed. We define evaluation criteria which are more adequate than pure edit distance and we describe how the measurement along these quality criteria is performed semi-automatically in a fast, convenient and above...

MT Evaluation Methodology Document - 2 Machine Translation Evaluation Methodology

MT-ComparEval: Graphical evaluation interface for Machine Translation development

The tool described in this article has been designed to helpMT developers by implementing aweb-based graphical user interface that allows to systematically compare and evaluate various MT engines/experiments using comparative analysis via automatic measures and statistics. The evaluation panel provides graphs, tests for statistical significance and n-gram statistics. We also present a demo serv...